{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How do I create my own dataset?\n",
    "\n",
    "So Caffe2 uses a binary DB format to store the data that we would like to train models on. A Caffe2 DB is a glorified name of a key-value storage where the keys are usually randomized so that the batches are approximately i.i.d. The values are the real stuff here: they contain the serialized strings of the specific data formats that you would like your training algorithm to ingest. So, the stored DB would look (semantically) like this:\n",
    "\n",
    "key1 value1\n",
    "key2 value2\n",
    "key3 value3\n",
    "...\n",
    "\n",
    "To a DB, it treats the keys and values as strings, but you probably want structured contents. One way to do this is to use a TensorProtos protocol buffer: it essentially wraps Tensors, aka multi-dimensional arrays, together with the tensor data type and shape information. Then, one can use the TensorProtosDBInput operator to load the data into an SGD training fashion.\n",
    "\n",
    "Here, we will show you one example of how to create your own dataset. To this end, we will use the UCI Iris dataset - which was a very popular classical dataset for classifying Iris flowers. It contains 4 real-valued features representing the dimensions of the flower, and classifies things into 3 types of Iris flowers. The dataset can be downloaded [here](https://archive.ics.uci.edu/ml/datasets/Iris)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.\n",
      "WARNING:root:Debug message: No module named caffe2_pybind11_state_gpu\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "If you are using homebrew leveldb on a Mac OS, you might see an error warning you that malloc_zone_unregister() failed. This is not a caffe2 issue but is due to the homebrew leveldb having an incompatible memory allocator. It does not affect usage.\n"
     ]
    }
   ],
   "source": [
    "# First let's import a few things needed.\n",
    "%matplotlib inline\n",
    "import urllib2 # for downloading the dataset from the web.\n",
    "import numpy as np\n",
    "from matplotlib import pyplot\n",
    "from StringIO import StringIO\n",
    "from caffe2.python import core, utils, workspace\n",
    "from caffe2.proto import caffe2_pb2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Raw data looks like this:\n",
      "5.1,3.5,1.4,0.2,Iris-setosa\n",
      "4.9,3.0,1.4,0.2,Iris-setosa\n",
      "4.7,3.2,1.3,0.2,Iris-setosa\n",
      "4.6,3.1,1.5,0.2,...\n"
     ]
    }
   ],
   "source": [
    "f = urllib2.urlopen('https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data')\n",
    "raw_data = f.read()\n",
    "print('Raw data looks like this:')\n",
    "print(raw_data[:100] + '...')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# load the features to a feature matrix.\n",
    "features = np.loadtxt(StringIO(raw_data), dtype=np.float32, delimiter=',', usecols=(0, 1, 2, 3))\n",
    "# load the labels to a feature matrix\n",
    "label_converter = lambda s : {'Iris-setosa':0, 'Iris-versicolor':1, 'Iris-virginica':2}[s]\n",
    "labels = np.loadtxt(StringIO(raw_data), dtype=np.int, delimiter=',', usecols=(4,), converters={4: label_converter})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we do training, one thing that is often beneficial is to separate the dataset into training and testing. In this case, let's randomly shuffle the data, use the first 100 data points to do training, and the remaining 50 to do testing. For more sophisticated approaches, you can use e.g. cross validation to separate your dataset into multiple training and testing splits. Read more about cross validation [here](http://scikit-learn.org/stable/modules/cross_validation.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "random_index = np.random.permutation(150)\n",
    "features = features[random_index]\n",
    "labels = labels[random_index]\n",
    "\n",
    "train_features = features[:100]\n",
    "train_labels = labels[:100]\n",
    "test_features = features[100:]\n",
    "test_labels = labels[100:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEKCAYAAAD3tSVSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYHHWd7/H3F2JYlcCAuqwhJjPkeFlgubkHgYiZLGYF\n4oXnWVclyYKrRyBcAsaDcsBswolxOeKGwMEV0dWDEl0ux0VOArpRMrmdw2WBECToo0lPgkQiLJmE\n2xIu3/NH1WR6erqnq7uru35d/Xk9zzzTXVVd9a3f9Hy7+lf1rZ+5OyIikk/7ZB2AiIg0j5K8iEiO\nKcmLiOSYkryISI4pyYuI5JiSvIhIjinJN5GZ7WNmz5nZhDSXTSGuU8ys0OztVNj2vmb2uplNjJ9/\n28wuS2ndPWa2u+j5WjM7K411x+v7mZmdmdb6RtnOn5rZBjPbZWbnNXt7eZHl+zpkSvJF4iS7O/55\nzcxeLJpW8z+3u7/u7uPc/XdpLpuSRAUSZvZZM1vVrG27++fc/aoEcTxhZh8YdaXuBXc/II0AzWyR\nmX23ZP0fcvcfpbH+Kr4E/MzdD3T3GxpZUdofdAm3eZyZPWhmL5jZfWb2Zy3cfMX3tZktNrNHzewV\nM7u8hTFlSkm+SJxkD4gTxVZgRtG0Ef/cZrZv66NsOSPhB0KN60x3hfn6W0wCHss6CKi9Xc1sLHAH\n8E9AF/DPwB2B/H1+DXwBuDvrQFrK3fVT5gcoAH9RMm0R0Zv2h8Au4CzgBOD/ATuBJ4FrgX3j5fcF\nXgcmxs9/EM+/C9gNrAcm1bpsPP80ojftTuA6YB1wVoV9eWO8vmeBR4EvAluK5l8BbI638yjwkXj6\nkcBLwCvAc8Af4ukfAR6O26Af+HKVtrwM+D3wBPAZ4LWS/fy7+PHbgBXxPv070BdP/2H8mhfiGC8B\nJsft9WmiD+SfD04r2u5a4CvAA/E6/zdwYDzvFKBQEucTwAeAGcDL8c9zwANF6zsrfmzA38X7/xTw\nXWD/eN5gbH8Tr3MH8KWE77vVcXu/FO9rN7AfsATYFrfj9cDYePmD4zb7Q9xmdwJvj+ddBbwKvBiv\na0lpG5XZr8/GMVwbr2/wb/NfgMfjaSuACRXiP61Mu/6Okv+lonkV30vV2pEq7+tR2vhHwOVZ55hW\n/ehIvnZnADe7+4HALUT/kHOJ/tmmAB8Czi1avvQo+EyipHoQ0Rt3Ua3Lmtkfx9v+AvBWog+k/zxK\nzIuAQ4kSxunA2SXzfw2c6NE3mMXAD83sbe7+S+BCYK1H32j+OF7+OWBm3AYfAeaa2enlNmxmHwYu\nAnqBdxG1TyWXEn3YvAU4BPgygLvPBLYDp3r0rWpp0WtOBt5NlJhhZBv+DTAbGE/0zbX4tWW/obj7\nCuBrwLJ4v8u17eeAmUQfCpOJ/v7XlSxzInAYcCpwpZlNLr/bw7Y9FbgXODfe137g60RH90cC7yT6\nO14Rv2Qf4EZgQrzMHqIEjbtfRnQAMriueaPtd5GTiL5JvBX4H2b2V0TvtY8QfRDfR/TBW84RwMaS\naRvj6eUkeS9Vasdq72tB3TX1WOfudwG4+8vu/qC7P+CRfuDbwNSi5Uu7Jm5394fd/TVgGXBMHcvO\nAB529+Xu/pq7X0N0hFXJXwOL3H23uz9BdCS4l7vf7u5/iB//M9ER1Z9XWpm797n74/HjR4k+cKZW\nWPyvgX9y91+7+0vAwlHifIUoGXe7+6vuvq5kfmn7ONGR5n+4+8sV1nlT0bb/DvjkKNuvxUzg6+6+\nzd1fAC6PpxXHtsDdX3H3h4mS5tG1bsTMjOgo+pL47/c80RH6mQDu/oy7/8Td9xTNK/1b1No9ttXd\nb4zf0y8THbR81d1/6+6vA18Fjjezt5d57f5ER+XFdgPjym0owXtptHYc9X0tESX52j1R/MTM3m1m\ny83s92a2C7iS6AiokqeKHr9I9E9R67LjS+Mg+kpcydtL5m8tnmlmn46v5njWzHYSHRlX3AczO9HM\nVpnZH8xsgOgrfqXlS2PdSuWk8/dEXRK/MLPfmNl/HWWfAHD3J6ssUrrt/czsoGrrTWA8w9txKzDW\nzN5WFNvTRfOr/a0r+ROi7ppH4r/Ps8By4vY2szeb2XfMbGv8t/gFo7//kih9b00CvlG0/aeJuoHK\nXQn2PFB68vtAoiP2EZK8l0Zpx1Hf1xJRkq9d6VfdbxH1Bx4Wf+VcQBNOLJb4PfCOkmmH1rD8pMEH\nZtYD/CPRV/qD3f0gou6bwX0o99X+R8BtwKHu3kV0kq3SPpfbdqVukufdfZ679xB1i33JzE4eJY4k\nSrf9srvvJOrff9PgDDMbQ9RNtDecKuvdTlE7Fq376QrL12sH0bmBd8d/n4PdvcvdD47nXxpv+8/j\nv8VflLy+dD9eADCzPyqa9idVXrMN+GzR9g9y9/3d/YEy8Zb7xvJnVD6RXMt7qdRTVHhfyxAl+caN\nA3a5+0tm9qcM749vluXAsWY2I77u/BJGP3q7DbjczA6Mr0+/oGje/kQnt56J1/U54D1F83cAE+Ik\nWPyane7+ipmdAHxqlG3fCnwm/sbzZqIuk7LM7MNmdlj89Dmio8XXi+I4rPQl5VZT8vysom0vJOoO\nAPgVMM7Mpsf7tgAo3scdRH29lfwImGdmk8xsHNEJ3uJ+6oqJysw+aGavjLLuveLuke8A15rZ4NH7\nBDObHi8yjujodpeZvSXej2LD2s3dnyJKjrPj2oxzqJ4cvwV82czeE2+/K+6nL+ceYF8zm2NmY83s\n80QfUqsrLF/tvTRawr+Vyu/rEcxsTPzhtg/wBjPbL+4OyzUl+cqSHjl+Afi0RUU43yS6+qbSeqqt\nM9Gycf/5J4FrgGeAHqIrFCr1Sy8g+sfuJ7oy4qaidT0K/E+iK1C2E53Yu7fotSuB3wA7zGx7PO18\n4Kq4e+oyhhJnuViXA98g+if/FfCvlZYl6ia6x8yeI7riY6m7r4/nfRX473GXwdzB1ZfbZMnjHwA3\nE135ZMDn47gGiE4If5/oK/8zDO8eu4Woa+dZM7u3aH2Dvh0vsxb4LVE/9CUV4ih9PoHoaqhKSl/7\nBaKuiPvjLo2fAv8pnreE6FLFf4/XuaLktUuBmfF+fD2edg7RiduniT4A7mUU7n478A/AbfH2NwB/\nWWHZl4GPEZ2Y3kl07uCj8XmlcuYw+ntptHas+L6u4HtEH4gfJzrYeDGOL9fMPVkuM7N9gH8Dfufu\nHy2ZNxX4CbAlnvRjd/9KmoFKZfHfZjvwV0VJUQIVF1n9wN3TLjITGWFM9UX2uhjYxMiTKoPWlCZ/\naR4z+xDREdh/AP+N6NK5+zMNShJx989kHYN0jkTdNRbdT+V0or7BioulEpEk9X6ib047gOnAGe6e\nqJ9XRDpH0j75a4jO4o/Wt3NifBneCjM7vPHQZDTuPt/d3xJfaTHF3R/KOiYRCU/VJG9mM4Ad7r6B\n6Gi93BH7g0Rl6scQFSTckWqUIiJSl6onXs3sq0Rl4a8S3StiHNGJ1Yp3trPodp/vdfdnS6anfaMr\nEZGO4O51dYlXPZJ398vdfaK7H0Z0Des9pQnezA4penw80YfHs5ThAdywp9rPggULMo9BcSrOdo1R\ncab/04harq4ZxszOjXK23wh83MzmMHT3vLTuDyIiIg2oKcm7+2riyjV3/1bR9G8QFbyIiEhAVPFa\nRm9vb9YhJKI409UOcbZDjKA4Q5K44jWVjZl5K7cnIpIHZoY368SrSCZWrICBgeHTBgai6SKSmJK8\nhGnKFLjiiqFEPzAQPZ8yJdu4RNqMumskXIOJ/dJL4eqrYfFi6OrKOiqRlmuku0ZJXsLW3w89PVAo\nQHd31tGIZEJ98pJPAwPREXyhEP0u7aMXkaqU5CVMg101ixdHR/CLFw/voxeRRNRdI2FasSI6yVrc\nBz8wAOvXw4wZ2cUlkgH1yYuI5Jj65EVEpCwleRGRHFOSFxHJMSV5EZEcU5IXEckxJXkRkRxTkhcR\nyTEleRGRHFOSFxHJMSV5aQ4N+iESBCV5aQ4N+iESBN27RppHg36IpEI3KJNwadAPkYbpBmUSJg36\nIZI5JXlpDg36IRIEdddIc2jQD5HUqE9eRCTH1CcvIiJlKcl3IhUqiXQMJflOpEIlkY6hPvlOpUIl\nkbahE69SHxUqibQFnXiV2qlQSaQjKMl3IhUqiXQMddd0IhUqibQV9cmLiORYS/rkzWwfM3vIzO6s\nMP86M/uNmW0ws2PqCUZERNJVS5/8xcCmcjPM7DRgsru/EzgXuCGF2ERGp6IukaoSJXkzmwCcDnyn\nwiIfA74P4O73AQea2SGpRChSiYq6RKpKeiR/DXApUKlD/VDgiaLnT8bTRJqnq2voyqD+/qErhlTU\nJbLXmGoLmNkMYIe7bzCzXqCuzv9BCxcu3Pu4t7eX3t7eRlYnna6rK6raHSzqUoKXHOjr66Ovry+V\ndVW9usbMvgrMBl4F3giMA37s7mcVLXMDsMrdb4mf/wqY6u47Stalq2skXbo9g3SApl5d4+6Xu/tE\ndz8M+BRwT3GCj90JnBUHcwIwUJrgRVKnoi6RququeDWzc83sHAB3vwsomNlvgW8B56cUn0hl69cP\nP3If7KNfvz7buEQComIoEZHA6QZlIiJSlpK81E5FSCJtQ0leaqciJJG2oT55qY8uXRRpGd2FUrKh\nkaVEWkInXqX1NLKUSFtQkpfaqQhJpG2ou0Zqp5GlRFpKffIiIjmmPnkRESlLST5vZs6EjRuHT9u4\nMZouIqkp9BeYPXc20z49jdlzZ1PoL2SyjmrUXZM3GzfCtGmwahUcddTI5yLSsEJ/gekXTmfz0Zth\nLLAHJj8ymZXXr6Snuyf1dai7RoYcdVSU0KdNg9tuU4IXaYL5S+YPJWeAsbD56M3MXzK/petIourI\nUNKGjjoKbrgBPvEJuPVWJXiRlD25+0l4S8nEsbB99/aWriMJHcnn0caNcN55UYI/77yRffQi0pBD\nDzgU9pRM3APjDxjf0nUkoT75vFGfvEjTtVOfvJJ83sycCZddNjyhb9wIV10FP/xhdnGJ5Eyhv8D8\nJfPZvns74w8Yz6J5ixIn+FrXoSQvIpJjurpGRETKUpLPmzRGbaq2Do0MJdI2lOTzJo1Rm6qtQyND\nibQN9cnnURqjNlVbh0aGEmkZnXiVkdIYtanaOjQylEhL6MSrDJfGqE3V1qGRoUTag7u37CfanDTV\nzp3u558f/S73PI11pLENEUkszp115V111+RNGqM2VVuHRoYSaSn1yYuI5Jj65EVEpCwl+aRCKQAK\nJQ6RJmnFaEmdREk+qVAKgEKJQ6QJBu/MuGzcMvp6+lg2bhnTL5yuRN8A9cnXIpQCoFDiEEnZ7Lmz\nWTZu2dBoSQB7YNZzs7j5upsziytrjfTJa2SoWnR1RYl1sAAoq8QaShwiKWvVaEmdRN01tQilACiU\nOERS1qrRkjqJumuSGuwiGewaKX3eaXGINEEaIy7lka6Tb4VQCoBCiUOkSdIYcSlvmprkzWw/YA3R\n5+oY4HZ3v7JkmanAT4At8aQfu/tXyqyrfZO8iEhGmnri1d1fNrNp7v6ime0LrDezu939/pJF17j7\nR+sJQkREmiPRiVd3fzF+uB/RB0O5w/G6PmUkZaEUS4USh0iHS5TkzWwfM3sYeApY6e4PlFnsRDPb\nYGYrzOzwVKOU5EIplgolDpEOV9OJVzM7ALgDuNDdNxVN3x94Pe7SOQ241t3fVeb16pNvhVCKpUKJ\nQ6TNtfTqGjObD7zg7ktGWaYAvNfdny2Z7gsWLNj7vLe3l97e3pq2LwmFMmpTKHGItJG+vj76+vr2\nPr/yyiubenXNW4FX3H2Xmb0R+BlwlbvfVbTMIe6+I358PHCru3eXWZeO5FshlCPoUOIQaXPNvtXw\n24FVZrYBuA/4mbvfZWbnmtk58TIfN7Nfxv32S4FP1hOMpKC4OKq7O/pd3DfeaXGIdDgVQ+VNKMVS\nocQhkgOqeBURyTGNDCUiImUpySeVRnHPwoWwdevwaVu3RtNbqdF9CWU/UlZ0MUNNNJKRBM3dW/YT\nba5N7dzpfv750e9yz5Po73c/8sjod7nnrdLovoSyHylbsKD212wpbPHJMyY7l+MsxLkcnzxjsm8p\nbEk9Pulcce6sL+/W+8K6NtbOSd59KBkWCrUn+EGDCXHt2mwTY6P7Esp+pKieJD/rollDCX7hUKKf\nddGs1OOTztVIktfIULVIY0SmSZPgm9+Ek0+GtWuj51lodF9C2Y8G9fUNddNcWXRv1d7e6KcajWQk\noVOSr0XpiEz1FPds3Qpz5kSJcc4cWL48mwTZ6L6Esh8NKk3mtZ5W2DuSUcmYpBrJSIJR71eAen5o\n5+4a9ckPCWU/UqY+eQkV6pNvgeXLRybBnTuj6UktWDAyEfb315ddGtHovoSyHylbtaq+120pbPFZ\nF83yaWdP81kXzVKCl9Q1kuRVDCUiEjgVQ4mISFlK8u2mWiGTRmQK0mDB1LFnqGBKWktJvt1UG3FJ\nIzIFp9BfYPqF01k2bhkbju1j2bhlTL9wuhK9tIT65NtRtfu06z7uQZk9dzbLxi0bcZnlrOdmcfN1\nN2cWl7SPRvrkdZ18O6pWyJRG0Zak5rFtT8KxJRPHwqZtKpiS5lN3TTsqLWQq1wc/2nxpqSMmxgVT\nxfbA4RNVMCXNp+6adlM84lJXV+3PpeUG++Q3H7056rLZA5MfmczK61fS092TdXjSBjRoSCepNuKS\nRmQKUqG/wPwl89m0bTuHTxzPonmLlOAlMSV5EZEcUzGUiIiU1TlJvhVFQipECla9oz61SrXRpVo1\n+lQIo1yFEEOu1HvTm3p+yPIGZWncRTKEbUhdQr53WrU7WbbqTpch3FEzhBhCRAM3KOucI/murugK\nkyuugP7+5lxx0optSO7MXzJ/6MobgLGw+ejNzF8yP9H8VsXRCiHEkDedVQzViiIhFSIFo9FRn1ql\n2uhSrRp9KoRRrkKIIW86K8mnMbJTCNuQRBod9alVqo0u1arRp0IY5SqEGPKmc7priouCuruHulXS\nrAZtxTYkdxbNW8TkRyYPVcXGxVKL5i1KNL9VcbRCCDHkTedcJ9+KIiEVIgWrry+sLppSg8VS23dv\nZ/wBI4ulqs1vVRytEEIMoVExlIhIjqkYSkREylKSTyqNQqck65g5EzZuHL7Mxo3RdMmtVhYALV1a\n/2vXrFtDz0k97P/eLnpO6mHNujXpBSbNUe8F9vX8kGUxVKPSKHRKso5HHnE/+ODod7nnkjutLgCa\nOrW+161eu9rHHDdmWJxjjhvjq9euTjU+GYkGiqHUJ1+LNEZcSrKOjRth2jS44QY47zxYtQqOOiq9\n/ZCgtHrkqN7e+m7z0HNSD/3T+kfE2b2qm8L/1a0HmkkjQ7VKGoVOSdZx1FFRgv/EJ+DWW5Xgc64V\nBUBLl8Idd0SPV68eutLojDPgkkuSrePpl3cOT/AAY+GZl3WJcMjUJ1+LNEZcSrKOjRujI/hbb41+\nl/bRS67sLQAqlnIB0CWXDFUAT5069Dhpggd4234HlY3zrfup2C9o9fbz1POD+uTVJy8jqE9eqqGB\nPnkl+aSWLx+Z0HfujKanuY4zzxyZ0B95JJouubWlsMVnXTTLp509zWddNKupd1285pr6X7t67Wrv\nPrHb9z+uy7tP7FaCb5FGknzVE69mth+whqg3bgxwu7tfWWa564DTgBeAT7v7hjLLeLXtiYjIcE09\n8eruL5vZNHd/0cz2Bdab2d3ufn9RAKcBk939nWb2PuAG4IR6AhIRkfQkOvHq7i/GD/cj+mAoPRz/\nGPD9eNn7gAPN7JC0gkxFCKM2LVwIW7cOn7Z1a7i3R0wgjRGXqq2jkW0MFhkde0blIqOkhUhZji6V\nRrFUmm3RiFBGwWpUu8SZtC99H+BhYDfw92Xm/x/gpKLnPweOK7NcM7qrkglh1Kb+fvcjj4x+l3ve\nhtIYcanaOurdRpITmrWc9MxqdKk0Tsym3RbN2pd2GRmq1XHS7JGh3P11dz8WmAC8z8wOT/vDpulC\nGLVp0iRYvhw+/GFYty76vXx5NF1Sl2SUoXYYiSiNGENpi1BGwWpUu8QJNRZDuftuM1sFnApsKpr1\nJPCOoucT4mkjLCzqmujt7aW3lfd/DWHUpkmT4JvfhJNPhrVr2zLBpzHiUrV1pLGNx7Y9CceWTBwL\nm7YNFRlVK0QKYXSpNIql0miLNIQyClajmh1nX18ffSn1D1ZN8mb2VuAVd99lZm8EpgNXlSx2J3AB\ncIuZnQAMuPuOcutbmGX/cwijNm3dCnPmRAl+zpy2PJJPY8SlautIYxtHTDyUDWVGGTp84lCRUbWR\niEIYXSqN0ZLSaIs0hDIKVqOaHWfpAfCVV464oDGxJN01bwdWmdkG4D7gZ+5+l5mda2bnALj7XUDB\nzH4LfAs4v+6ImiWEUZu2bh3qonn/+4e6bkpPxkoqkowy1A4jEaURYyhtEcooWI1qlziBDiqGSqOY\nqVELFow8ydrfn90ZvRSsWtX8dTSyjcEio2M/VrnIKGkhUhr7Wq80iqXSbItGVNtGKwvDGtHKONFd\nKEVE8ksjQ0lba+Z18rVoNI7R5rfNNdWSO0rykrm8J/lCf4HpF05n2bhl9PX0sWzcMqZfOF2JXlpC\nSV6kydrpmmrJHw0aIploxXXyrYgjSZztcu235JOSvGSiFdfJtyKOJHG2y7Xfkk/qrhFpsra6plpy\nR0leMlet+6VVtw9oNI5K83u6e1h5/UpmPTeLaYVpzHpuFiuvX0lPd08dUYrURtfJi4gETtfJi4hI\nWUrykpl2GKyj2GjXwVfbj2rLrFm3hp6Teuh6Xxc9J/WwZt2ammKQ4UIY/CQUurpGMjFYILT56M3R\n5YV74N4L7y3bV93X17p++dGUiyPJflRbZs26NZxy8Sm8euqrMBZ27dnFKRefwi+u/QUfeP8HqsYg\nw9Xy3gp5G2nRkbxkIi8FQmkMxnH2F8/em+AH57966quc/cWzW7YfeRLC4Cch0ZG8ZKIdButIEkeS\nQqdqy+x8befwa+jj+QOvDSSKQYYLYfCTkCjJSybaYbCOJHEkKXSqtsxB+x7Erj27Rszv2rcrUQwy\nXAiDn4RE3TWSibwUCKUxGMdNX7uJMT8dM2z+mJ+O4aav3dSy/ciTEAY/CYmuk5fMFPoLzF8yn+27\ntzP+gPEsmreo7EmrUE42VoojyX5UW2bNujWc/cWzGXhtgK59u7jpazeNOOk6WgwyXNL3VujbGNTI\ndfJK8iIigVMxlIiIlKUk3+FCKK5pJIakRUSha5fCGmk/SvIdrp2T/GARUf+0fnadvov+af2ccvEp\nbZfoNXKUNJOSvLStvBQRtVNhjbQfXSffgUIorkkjhmpFRO2inQprpP0oyXegEIpr0oihWhFRu2in\nwhppP+qukbaVlyKidiqskfaj6+Q7XAjFNY3EkLSIKHStLKyR9qNiKBGRHFMxlIiIlKUkL03X6LX4\nSV4fwvX+IiFSkpemU5IXyY6SvIhIjuk6eWmKRoudkrw+hKIukdApyUtTNFrslOT1IRR1iYRO3TUi\nIjmmJC9N12jXSZLXq3tGpDwVQ4mIBK6pxVBmNsHM7jGzx8zsUTObW2aZqWY2YGYPxT9fricYERFJ\nV5LumleBee5+BHAicIGZvafMcmvc/bj45yupRtlh0holKJRrx0OJIwRZtoVGn+pMVZO8uz/l7hvi\nx88DjwOHllm0rq8SMlyaowSFklxDiSMEWbWFRp/qXDWdeDWzbuAY4L4ys080sw1mtsLMDk8hto6k\nUYKkGfS+6lyJr5M3s/2B24GL4yP6Yg8CE939RTM7DbgDeFe59Swsupi5t7eXXl0WMUyjowSFUiAU\nShwhCKEtNPpUe+nr66Mvpa99iZK8mY0hSvA/cPeflM4vTvrufreZ/aOZHezuz5Yuu1AVK6NqdJSg\nUAqEQokjBCG0hUafai+lB8BXFh8d1Chpd813gU3ufm25mWZ2SNHj44kuzRyR4KU6jRIkzaD3Veeq\nep28mU0B1gCPAh7/XA5MAtzdbzSzC4A5wCvAS8Dn3X1Ev72uk08mrVGCQhj1CcKJIwRZtoVGn2pf\nGhlKRCTHNDKUiIiUpSQvTbd0adYRiHQuJXlpujvuyDoCkc6lJC8ikmMaNESaYunSoSP41auHrig5\n4wy45JLMwhLpOLq6Rpqut1f3rxFphK6uERGRspTkpenOOCPrCEQ6l7prREQCp+4aEREpS0l+0IoV\nMDAwfNrAQDRdJCU6AS2tpiQ/aMoUuOKKoUQ/MBA9nzIl27gkV5TkpdWU5Ad1dcHixVFi7++Pfi9e\nHE0XEWlTKoYq1tUFl14KPT1QKCjBSypCGBlKOpeSfLGBAbj66ijBX321juQlFSGMDCWdS901gwb7\n4Bcvhu7uoa6b0pOxIiJtRNfJD1qxIjrJWnzkPjAA69fDjBnZxSW5olGypB4aGUpEJMdUDCUiImUp\nyUvHK/QXmD13NseeMY3Zc2dT6C9kHZJIatRdIx2t0F9g+oXT2Xz0ZhgL7IHJj0xm5fUr6enuyTo8\nEUDdNSJ1m79k/lCCBxgLm4/ezPwl8zONSyQtSvLS0R7b9uRQgh80FjZt255JPCJpU5KXjnbExENh\nT8nEPXD4xPGZxCOSNvXJS0dTn7y0A10nL9KAQn+B+Uvms2nbdg6fOJ5F8xYpwUtQlORFRHJMV9eI\niEhZSvIiIjmmJC8ikmNK8iIiOaYkLyKSY0ryIiI5piQvIpJjSvIiIjlWNcmb2QQzu8fMHjOzR81s\nboXlrjOz35jZBjM7Jv1QRUSkVkmO5F8F5rn7EcCJwAVm9p7iBczsNGCyu78TOBe4IfVIW6ivry/r\nEBJRnOlqhzjbIUZQnCGpmuTd/Sl33xA/fh54HDi0ZLGPAd+Pl7kPONDMDkk51pZplz+84kxXO8TZ\nDjGC4gxJTX3yZtYNHAPcVzLrUOCJoudPMvKDQEREWixxkjez/YHbgYvjI3oREQlcortQmtkYYDlw\nt7tfW2b+DcAqd78lfv4rYKq77yhZTregFBGpQ713oRyTcLnvApvKJfjYncAFwC1mdgIwUJrgGwlS\nRETqU/UoaJoGAAAD0UlEQVRI3symAGuARwGPfy4HJgHu7jfGy10PnAq8APytuz/UxLhFRCSBlg4a\nIiIirdW0ilcz28fMHjKzOyvMz7x4arQYzWyqmQ3E8x8ysy9nEWMcS7+ZPWJmD5vZ/RWWCaE9R40z\nhDY1swPN7DYzezwu8HtfmWVCaMtR4wykLd8V/60fin/vKlcsmXV7JokzhPaM4/i8mf3SzDaa2TIz\nG1tmmdra092b8gN8HrgZuLPMvNOAFfHj9wH3NiuOBmKcWm56RnFuAQ4aZX4o7VktzszbFPhfRN2J\nEJ2TOiDQtqwWZ+ZtWRLPPsB24B0htmeCODNvT2B8/D80Nn5+C3BWo+3ZlCN5M5sAnA58p8IimRdP\nJYgRIJQTxcbo37oyb89YtTgHl8mEmR0AnOzu3wNw91fdfXfJYpm3ZcI4IZz3J8AHgc3u/kTJ9Mzb\ns0SlOCGM9twXeHN8ReObiD6QitXcns3qrrkGuJToJG05IRRPVYsR4MT4K9EKMzu8RXGV48BKM3vA\nzD5XZn4I7QnV44Rs27QHeMbMvhd/Jb/RzN5YskwIbZkkTgjn/QnwSeBHZaaH0J7FKsUJGbenu28H\n/gHYRtROA+7+85LFam7P1JO8mc0Adnh0KwQjjE/HYRLG+CAw0d2PAa4H7mhhiKWmuPtxRN88LjCz\n92cYy2iqxZl1m44BjgO+Ecf5InBZi2NIIkmcWbflXmb2BuCjwG1ZxZBElTgzb08z6yI6Up9E1HWz\nv5nNbHS9zTiSnwJ81My2EH1iTjOz75cs8yTwjqLnE+JprVI1Rnd/3t1fjB/fDbzBzA5uYYzFsfw+\n/v008C/A8SWLZN2eQPU4A2jT3wFPuPu/xc9vJ0qmxUJoy6pxBtCWxU4DHoz/7qVCaM9BFeMMpD0/\nCGxx92fd/TXgx8BJJcvU3J6pJ3l3v9zdJ7r7YcCngHvc/aySxe4EzgKwUYqnmiVJjMX9XGZ2PNHl\nps+2Ksaibb/JoltKYGZvBv4S+GXJYpm2Z9I4s27TuE2eMLN3xZNOATaVLJZ5WyaJM+u2LHEmlbtA\nMm/PIhXjDKQ9twEnmNkfmZkR/d0fL1mm5vZMWvHaMDM7l7h4yt3vMrPTzey3xMVTrYpjNMUxAh83\nsznAK8BLRH15WTgE+BeLbgkxBljm7v8aYHtWjZMw2nQusCz+6r4F+NsA27JqnITRlpjZm4iOQM8p\nmhZce1aLkwDa093vN7PbgYfjOB4Cbmy0PVUMJSKSYxr+T0Qkx5TkRURyTEleRCTHlORFRHJMSV5E\nJMeU5EVEckxJXkQkx5TkRURy7P8DYWwpFkdES0wAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111528750>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEKCAYAAAD3tSVSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH4dJREFUeJzt3X+YHVWd5/H3F5ogPxIaUJlNwHSbxRl/zIrMDgqN0llF\ngTjK7o6Po+EBmXHM8GMAM8Z1gGwS24yjcRBZZkVWxagZVFhlkAhrRmgSGESRHzICj0i6AyYkiOQa\nMIwh4bt/nHPT1dV1+97uvn1vVd3P63nu07eqzq361qnq7617quqUuTsiIlJO+7Q7ABERmT5K8iIi\nJaYkLyJSYkryIiIlpiQvIlJiSvIiIiWmJN9mZjbDzJ41s99rwbLeYWaPTvdyaix7fzN70cxmx+Fr\nzOwjTZr30Wb2TGL4LjN7fzPmHef3AzN7T7PmN85yXmtmD5jZDjP74HQvryzauV8XgZJ8DTHx7oiv\nPWa2MzHufVOY76gE5O673H2mu29tTuR1NXRjhJktMrN107Vsdz/b3T/TQBxPmtkJ487U/VF3P6wZ\nAZrZJ83s6tT83+ru1zVj/nX8LfBdd5/l7l+cyoya/UXX4DL/2MzuM7PnzOyHZvbaFi6+5n5tZn9v\nZg+a2W4z+2gLY8oFJfkaYuKd5e6zgE3AgsS4a9sdX4s0+045a/L8MLN9mz3PNpoLPNTuIGDi9Wpm\nLwFuAK4CDgWuB75jZnnIMY8Ai4HvtzuQtnB3veq8gCHgv6TG7QMsBR4DngK+BsyK0w4ErgV+DWwH\n7gIOAT4D7AZ2AjuAVcD+wIvA7PjZa4HPArfEMhuAoxLLXQD8HHgmlrsLeH+NuA8E1sQYHgA+Bvw8\nMX0psDEu56fAaXH8McDzwC7gWWBLHH86cD/wG2AY+Ns69XYJsBV4HPgLYE9qPS+O748Abo5xPg38\nSxz/rfiZ52KM5wO/D7wAfDDO95bquMRy7wI+DtwT53kdMDNOewfwaCrOJ4ETgHcDv4uvZ4EfJub3\n/sR2X0H44n8S+CJwUJxWje0DwBPANuAjDe5jd8bPPh/X9UjgJcDlcT23AJ8D9ovlXwp8j7DvPU1I\nsEfEaVn72ag6ylivRcAPgCsJ+9bFifGPxGV8t7r9MuL/E+AXiWGL9fOWGuVr7kv16pE6+/U4dXwd\n8NF255NWv9oeQBFeZCf5/wHcTkhQM4AvA1+K0y4gJKgZMSn8EXBAnHYX8L7EfPZnbPLbCrwe2Dfu\nmF+O0/5DTD6nxmlLCAmpVpK/HFgHzCQcJT7C6CT/HuDl8f3CmBAOi8OLgO+n5jcfeHV8f0z8x397\njWWfHv9Bj47/lNdnrGc1kVxG+MIyoAs4MTGfJ4HjE8O/T/hSvJqQBPeP43YlytwVt1l12TcCV8dp\n70gnhLiME+L7T1bLpuZXTYbnAj8jJOGDCYnv6lRs/ytu+/8ct09Pg/tZet/4fNyPZsbXzcDSOO3l\nhMQ6I077DvBP48xrVB1lrNciwpf6n8ftsD/w3riu8+L+9nHg1hqxfwz4v6lx3wfOqVG+5r5Urx6p\ns1+PU78dmeTz8FOqqBYBH3P3be6+CxgA/ixOewF4GXC0u7/o7j9x9+cTn003W6SHv+XuD7j7HuCf\nCP8EAO8EfuTuN8dpnwEq48T4HuDj7v6su28C/jE50d2vc/en4vs1wGbCF1Imd7/N3R+O7+8n/NOc\nNM6y/4+H9vKdhKPfWl4AZhP+iXe7+x2p6en6cUKy+3d3/12NeV6TWPYyYNLnUVLeD6xy91+6+3OE\nXysLU7H9Tw/nWu4hJKD/NIH5G+xtLvlz4MK4/Z4FPkVcD3d/yt2/G5dTnZbeFhNtHtvo7l/24HeE\nffwT7v5Y3N8GgBPN7GUZnz2YcFSetIOQiMdoYF8arx7H3a9lNCX5yTsK+J6ZPROv7LgXwMwOA74E\nrAeuN7PHzWylmU3kHy55EnYn4R8IQiJ8ojrBw+HJ5qwZxOX9HvDLxOhNqTJ/Ea/meMbMthOO2F5a\nKygz6zOzQTN7yswqwFnjlB8Va1x2rTr4BOFo+jYz+7mZfbhWDNGL7r6tTpn0sg80s8yEM0GzGV2P\nm4CXmNmhcXiPu29PTE9uv4kuZz/gZ4l97AZifZvZwWb2JTPbFLfF/2OcbdegJ1LDc4GrEst/inC0\nf2TGZ58DZqXGHUL45TlGA/tSZj02sl/LaEryk/dLQhPOYfF1qLsf5O7PxKOPZe7+auAthCOP6lH+\nVE5mPkn4cgH2JvI5WQXjF8DWZHnCP231s0cDVwAfrMZPOL9QTcRZcX6T0Mwyx927gdXUTtyjYo3L\nzlz3eER2kbv3AP8duNTMjh8njkbqML3snfGI97eEJhwAzGw/IHllTr15byFRj/H986mE1AxPEn7h\nzEvsY93u/vI4/WOEbf9HcVu8ndHbIr0evwX2jetblb5sN/2Zx4EPpPbxg939vox4f0ZoYgT27puv\ni+OzTGRfGgkw7NfbqLFfy1hK8pP3BeBTZnYkgJm93MzeGd+/1cxeHXf05wgnwfbEz20DXjnJZd4I\nHGdmp8Sf838DdI9T/jrgEjObZWZzgXMS0w6OMT1tZl1m9lfAf0xM3wYcZWZdiXEHAc+4+wvxssbx\nrh3/FvDBeA37wYSTvJnM7E/MrDcOPkuorxfj8FbG1ldWMkiP+0Bi2cuAb8TxDwOHmdn8uG4rUp/d\nBvRS27XAR8zsqPjLYIBwEnC82MKEsN2erzU9yd13E87zXGFmh8fPH2Vmb4tFZhJPrJrZS4FLU7NI\n72dbgF8BC81sHzM7lxoHCAlfAJaa2avi8g81s/9Wo+w64AAz+5CZzQA+QtiW6aa3qnr70ngJ/1vU\n3q/HiPv3Swj5bj8L92w0/UqvvFKSb0zW0d2nCDv2rWb2G8LO/IY4bQ7wz4xctXKTu38rTvsscJaZ\n/drM/j5j/jWPJD1cS/8+wgmpXxF+0j9IOCmV5VLCFT6PE04Qrk7M6z7C5W4/ITT5zAV+nPjsLYSr\nHp4ys8fjuHOAf4jr+xHCP1utWG8gnBzdQLgs8JZ0kcT7VxOaanYAg4Q277vjtL8D/i42GZyb8dms\n+TnhaqdrCU0Qu2O8uPuvgQsJ5zqeICS/pxOf/QZwUFzeHYn5VX0e+Dbwr8Cj8bN/UyOO9PCR1E56\nWZ+9KMZ3T2zS+B6hSQ3C+ZiXEbbvemBt6rOj9jN3fxH4S2A5Yd+ZQ7j6qHYw7t8g7Gvfjsu/F3hb\njbL/Trg66RzCVS9/Cpwel5vlrxh/XxqvHmvu1zV8jfCFeDrh5PFOxj9AKRULv34aKBiud70H+KW7\nvytj+hWEqz5+S/iJd38zA5Wx4tH8VuCdiaQoOWVmXyWcjN7Q7likc3TVL7LXhYQjsvTJFczsVELb\n4dFm9kbCEeKbmhOiJJnZKYSjyF2EKzt+Szgal5xz9zPbHYN0noaaa2K782mEGz+yvBv4KkA8ojzE\nzI5oSoSS9hbCNeBbCdca/9fYfisiMkajbfKfJdx4U6ttZw6jL7/aTP2TOjIJ7n6xux8er7Q4scaV\nDiIiQANJ3swWANtiG7sxDf2PiIjI9GikTb4PeJeZnQYcAMw0s6+m2hc3M/q61SPJuEnHzJrd4ZWI\nSEdw90kdYNc9ko/NA69w91cSbui5NeME0o3AmQBm9iagUuuORG9xvw2TeS1btqztMShOxVnUGBVn\n819TMZGra0Yxs0UhZ/vV7v49MzvNzH5BuNrj7ClFJSIiTTGhJO/utxN6XsTdv5Cadn4T4xIRkSbQ\nHa8Z+vv72x1CQxRncxUhziLECIozTxq+47UpCzPzVi5PRKQMzAyfrhOvIiJSXEryIiIlpiQvE7d2\nLVRSD6SqVMJ4EckVJXmZuL4+uOSSkURfqYThvr72xiUiY+jEq0xONbEvWQKrVsHKldA93vNLRGSy\npnLiVUleJm94GHp7YWgIenraHY1IaenqGmm9SiUcwQ8Nhb/pNnoRyQUleZm4alPNypXhCH7lytFt\n9CKSG2qukYlbuzacZE22wVcqcOedsGBB++ISKSm1yYuIlJja5EVEJJOSvIhIiSnJi4iUmJK85JO6\nThBpCiV5ySd1nSDSFLq6RvJLXSeIALqEUspMXSeI6BJKKSl1nSAyZUrykk/qOkGkKdRcI/mkrhNE\n9lKbvIhIialNXkREMinJi4iUmJK8iEiJKcmLiJSYkryISIkpyYuIlJiSvIhIiSnJi4iUmJK8iEiJ\nKcmLiJSYkryISIkpyYuIlJiSvIhIiSnJi4iUmJK8iEiJKcmLiJSYkryISIkpyYuIlJiSvIhIiSnJ\ni4iUmJK8iEiJKcmLiJSYkryISIkpyYuIlFjdJG9m+5vZ3WZ2n5k9aGbLMsqcZGYVM7s3vi6dnnBF\nRGQi6iZ5d/8dMN/d3wAcA5xqZsdlFF3v7sfG1yeaHahEy5fDpk2jx23aFMYDrF0Llcro6ZVKGF/V\nSBkRKYWGmmvcfWd8uz/QBXhGMWtWUDKOs8+Gd75zJNFv2hSGzz47DPf1wSWXjCTxSiUM9/WNzKOR\nMiJSDu5e90X4MrgP2AF8MmP6ScDTwP3AWuA1Nebj0gTDw+6ve537hg3h7/Dw6Onbt7ufe6770FD4\nu3372Hk0UkZEciHmzobydfpl4fONMbNZwA3A+e7+UGL8wcCL7r7TzE4FPufur8r4vC9bNtKk39/f\nT39//4S+lCS64w5485thwwY48cSx04eHobcXhoagpyd7Ho2UEZGWGxwcZHBwcO/wihUrcPfJtZZM\n9FsBWAosrlNmCDgsY/w0fc91GB3Ji3QUpnAk30hSfylwSHx/ALAeOC1V5ojE++OA4Rrzmu66KL9q\ngq8m9vRwNXlXk3Z6uNEyIpIbU0nydZtrzOwPgdWEdvl9gG+6+0ozWxQXfLWZnQecA7wAPA982N3v\nzpiX11ue1LF8eTjJOnfuyLhNm+Caa8K0tWvDCdTu7pHplQrceScsWBCGGykjIrlhZpNurplQm/xU\nKcmLiEzcVJK87ngVESkxJXkRkRJTkhcRKTEleRGRElOSL5pm9F0jLTc0PMQZF5zB/A/M54wLzmBo\neKjdIUmHUJIvmmb0XSMtNTQ8xMnnn8yamWsY7B1kzcw1nHz+yUr00hK6hLKIqon985+Hc86Bm24a\nfd18NbEvWQKrVsHKlaOviZeWOuOCM1gzcw3MSIzcBQufXcjXr/h62+KS4pjKJZRdzQ5GWmDu3JDg\nq33XJBM8hIS+ZMlIvzRK8G21ecdmODw1cgZs2bGlLfFIZ1FzTRFt2hSO4DdsCH/TbfSVSjiCHxoK\nf9Nt9NJSc2bNgV2pkbtg9qzZbYlHOouaa4qm2lRTbaJJD1ebaqpNNOlhablqm/xjr38sNNnsgnkP\nzGPdlevo7eltd3hSAOrWoJM0o+8aabmh4SGWXraULTu2MHvWbAYWDyjBS8OU5EVESkx914iISCYl\neRGRElOSFxEpMSV5kQ6x/o719J7QS/cbu+k9oZf1d6xvd0jSAjrxKtIB1t+xnrde+FZ2n7J772Wc\nXbd08YPP/YC3nPiWdocndejqGhEZV+8JvQzPHx7TtULPbT0M/av60Mk7XV0jIuPavmf76AQPMAMq\ne3Q3dNkpyYt0gEP3PTSza4XufXUXdNkpyYt0gNWfXk3XLV0jiT62ya/+9Oq2xiXTT23yIh1i/R3r\nOeujZ1HZU6F7325Wf3q1TroWhE68ioiUmE68iohIJiV5EZESU5IXESkxJflGrV079glLlUoYP5Ey\nrYhDCmtwsN0RjG9oeIgzLjiD+R+YzxkXnFHzYeR5X49OoiTfqL6+8ISlaoKtPnGpr29iZVoRhxRW\nnpNj9QlXa2auYbB3kDUz13Dy+SdnJvo8r0enUZJvVHd3eITeJZfA8HD2I/UaKdOKOESmwdLLlo48\nwhBgBjz2+sdYetnStsYl4+tqdwCF0t0NS5ZAb294SHZWYm2kTCvikMIYHBw58l2xYmR8f3945cXm\nHZvh8NTIGbBlxxagOOvRaZTkJ6JSgVWrQmJdtSr7CLqRMq2IQwojnQSXL29TIHXMmTUn3DGb6uRs\n9qzZQHHWo9OouaZR1bbvlSuhp2ekySR5ErSRMq2IQ2QaDCweYN4D80Z1jTDvgXkMLB5oa1wyPt3x\n2qi1a8PJzeQRc6UCd94JCxY0XqYVcUhhDQ7mu2ljaHiIpZctZcuOLcyeNZuBxQP09vSOKZf39Sga\ndWsgIlJi6tZAREQyKcmLiJSYkryISIkpyYt0iEa7JJBy0YlXkQ5Q7ZJg7x2r8fLHdVeuy7w6RvJF\nJ15FZFzqkqBzKcmLdIDNOzaPvlMVRnVJIOWlJC/SAfZ2SZCU6JJAyktt8iIdQG3yxaY7XkWkrka7\nJJD8UZIXESkxXV0jIiKZlORFREqsbpI3s/3N7G4zu8/MHjSzZTXKXWFmj5rZ/WZ2TPNDFRGRiaqb\n5N39d8B8d38DcAxwqpkdlyxjZqcC89z9aGARcNV0BFvT2rVjH5pRqYTxRVNvXcq0rlHeH/qch+4A\n1t+xnt4Teul+Yze9J/Sy/o71NcvmvT6LIg/bvSncveEXcCBwD/DHqfFXAe9NDD8MHJHxeZ8W27e7\nn3tu+Js1XCT11qVM6xotW9buCGrbOLTR5y2Y51yMsxznYnzegnm+cWhjy2K4fcPt3nVs16gYuo7t\n8ts33J5ZPs/1WRR52O5JMXdOKF9XXw21yZvZPmZ2H7AVWOfuP04VmQM8kRjeHMe1Rnf3yGPwhodH\nHo9XxOee1luXMq1rAeShO4CzPnoWu0/ZPSqG3afs5qyPntWyGDpNHrZ7szT0IG93fxF4g5nNAm4w\ns9e4+0OTWeDyxNN9+/v76W/WM8K6u2HJEujtDQ+4LnLSq7cuJVjXwcGRZoUVK0bGpx8G3W6bd2yG\nw1MjW9wdwPY92zO7JKjsGWm2K0p9FkW7t/vg4CCDTWp3ayjJV7n7DjO7DTgFSCb5zcBRieEj47gx\nlk/XI9wrFVi1KiS9VauKfXRbb11KsK7p5DNdu8VU7e0OIJlkW9wdwKH7Hspvdv1mTAzd+45s86LU\nZ1G0e7unD4BXJL+5J6iRq2teamaHxPcHACcDj6SK3QicGcu8Cai4+7ZJRzVRlcpIs0VPz0hzRvoE\nZRHUW5cyrWsBDCweYN4D80b6fYndAQwsHmhZDKs/vZquW7pGxdB1SxerP726ZTF0mjxs96ap12gP\n/CFwL3A/8FPgkjh+EfChRLkrgV8ADwDH1pjX9JyVuOmmsScet28P44um3rqUaV2j225rdwTj2zi0\n0Rf+9UKff9Z8X/jXC9ty8u32Dbd7z/E93n1ct/cc31PzpKt7/uuzKPKw3auYwolXdWsgIpJz6tZA\nREQyKcmLiJSYkryISIl1TpIvYXcAUj7NuDS67N0alKa7gRbpnCTf15d9KWJfX3vjEklQkh9f9QlX\na2auYbB3kDUz13Dy+Scr0Y+jc5K8ugMQKbwydTfQKhO647XwStAdgJRPM7ok6JRuDdrd3UARdVaS\nL0F3AFI+zeiSoFO6NWh3dwNF1DnNNeoOQKTwStXdQIt0zh2va9eGk6zpjr7uvBMWLGhPTCIpg4NT\nb15pxjzybGh4iKWXLWXLji3MnjWbgcUD9Pb0tjusaTWVO147J8mLiBSUujUQEZFMSvIiIiWmJC8i\nUmJK8iIiJaYkL7k33m36rezHpMzdBaR10rqWnZK85F6thNPqfkw6KfF10rqWnZK8FJb6MRGpr7O6\nNZDCaKQvllb0Y9IpfcJAZ61rJ1GSl1xqpC+WVvRj0il9wkBnrWsnUXONFJb6MRGpT0lecq9WU0Fv\nTy/rrlzHwmcXMn9oPgufXci6K9dNWz8mndRk0UnrWnbqu0ZEJOfUd42IiGRSkhcRKTEleRGRElOS\nl1K4/PKpz2M67/JsZfcLIklK8lIKN9ww9XlMV5JvdfcLIklK8iLTTN0vSDvpjlcprMsvHzmCv/32\nkWu7Tz8dLrqosXm04lb+VnS/IFKLkrwU1kUXjSTz/v7JNbe04lb+VnS/IFKLmmtEppm6X5B2UpKX\nUjj99KnPY7pu5W919wsiSerWQEQk59StgYiIZFKSFxEpMSV5EZESU5IXqUNdEkiWouwXOvEqMo5q\nlwR771iNlz/q6pjO1ur9QideRaaJuiSQLEXaL5TkRcaxecfm0XeqgrokkELtF0ryIuPY2yVBkrok\n6HhF2i/UJi8yDrXJS5YitckryYvUMTQ8xNLLlrJlxxZmz5rNwOIBJXhp6X6hJC8iUmK6ukZERDIp\nyYuIlFjdJG9mR5rZrWb2MzN70MwuyChzkplVzOze+Lp0esIVEZGJaORIfjew2N1fCxwPnGdmf5BR\nbr27Hxtfn2hqlFJqtZ7oVJTbxhtRpnWRYqn7+D933wpsje+fM7OHgTnAI6mikzopIDI4OPaBHaMu\nUTsc2AU/PP+Hhbx0sUzrIsUzoTZ5M+sBjgHuzph8vJndb2Zrzew1TYhNOliRbhuvp0zrIsXT8IO8\nzexg4HrgQnd/LjX5J8Ar3H2nmZ0K3AC8Kms+yxNPSu7v76d/up65Jrk2ODjSTLNixcj46oO1N+/Y\nHI56k3J623g9ZVoXaY3BwUEGJ/Nk+gwNJXkz6yIk+K+5+z+npyeTvrvfbGb/28wOc/dn0mWTSV46\nVzWZV6V3i723jSf7B8npbeP1lGldpDXSB8ArkkdCE9Roc82XgYfc/XNZE83siMT74wg3WY1J8CKN\nGlg8wLwH5o30DxJvGx9YPNDWuCajTOsixVP3jlcz6wPWAw8CHl8XA3MBd/erzew84BzgBeB54MPu\nPqbdXne8SpasE69Qru4EyrQu0nrq1kBEpMTUrYGIiGRSkhcRKTEleRGRElOSz7EmXSYrIh1MST7H\nlORFZKqU5EVESqzhbg2kNerd7i8iMhFK8jlT73Z/EZGJUHONiEiJKcnnmJpnRGSq1K2BiEjOqVsD\nERHJpCQvIlJiSvIiIiWmJF9gRbkjtihxipSRknyBFSV5FiVOkTJSkhcRKTHd8VowRen2oChxipSd\nknzBFKXbg6LEKVJ2aq4RESkxJfkCK0qzR1HiFCkjdWsgIpJz6tZAREQyKcmLiJSYkryISIkpyYs0\nie7slTxSkhdpEiV5ySMleRGREtMdryJToO4bJO+U5EWmQN03SN6puUZEpMSU5EWaRM0zkkfq1kBE\nJOfUrYGIiGRSkhcRKTEleRGRElOSFxEpMSV5EZESU5IXESkxJXkRkRJTkhcRKTEleRGRElOSFxEp\nMSV5EZESU5IXESkxJXkRkRKrm+TN7Egzu9XMfmZmD5rZBTXKXWFmj5rZ/WZ2TPNDFRGRiWrkSH43\nsNjdXwscD5xnZn+QLGBmpwLz3P1oYBFwVdMjbaHBgjyRWXE2VxHiLEKMoDjzpG6Sd/et7n5/fP8c\n8DAwJ1Xs3cBXY5m7gUPM7Igmx9oyRdnwX/nKYLtDaEhR6rMIcRYhRlCceTKhNnkz6wGOAe5OTZoD\nPJEY3szYLwJpsuHhdkcgInnXcJI3s4OB64EL4xG9iIjkXEOP/zOzLuAm4GZ3/1zG9KuA29z9m3H4\nEeAkd9+WKqdn/4mITMJkH//X1WC5LwMPZSX46EbgPOCbZvYmoJJO8FMJUkREJqfukbyZ9QHrgQcB\nj6+LgbmAu/vVsdyVwCnAb4Gz3f3eaYxbREQa0FBzjYiIFNO03fFqZvuY2b1mdmON6W2/eWq8GM3s\nJDOrxOn3mtml7YgxxjJsZg+Y2X1m9qMaZfJQn+PGmYc6NbNDzOw6M3s43uD3xowyeajLcePMSV2+\nKm7re+Pf32TdLNnu+mwkzjzUZ4zjw2b2b2b2UzNbY2YzMspMrD7dfVpewIeBrwM3Zkw7FVgb378R\n+OF0xTGFGE/KGt+mODcCh44zPS/1WS/Ottcp8BVCcyKEc1KzclqX9eJse12m4tkH2AIclcf6bCDO\nttcnMDv+D82Iw98EzpxqfU7LkbyZHQmcBnyxRpG23zzVQIwAeTlRbIz/q6vt9RnVi7Napi3MbBbw\nZne/BsDdd7v7jlSxttdlg3FCfvZPgLcBj7n7E6nxba/PlFpxQj7qc1/goHhF44GEL6SkCdfndDXX\nfBZYQjhJmyUPN0/VixHg+PiTaK2ZvaZFcWVxYJ2Z/djM/jJjeh7qE+rHCe2t017gaTO7Jv4kv9rM\nDkiVyUNdNhIn5Gf/BHgvcG3G+DzUZ1KtOKHN9enuW4B/AB4n1FPF3f8lVWzC9dn0JG9mC4BtHrpC\nMPLx7ThKgzH+BHiFux8DXAnc0MIQ0/rc/VjCL4/zzOzENsYynnpxtrtOu4BjgX+Mce4EPtbiGBrR\nSJztrsu9zGw/4F3Ade2KoRF14mx7fZpZN+FIfS6h6eZgM3v/VOc7HUfyfcC7zGwj4Rtzvpl9NVVm\nM3BUYvjIOK5V6sbo7s+5+874/mZgPzM7rIUxJmN5Mv79FfAd4LhUkXbXJ1A/zhzU6S+BJ9z9njh8\nPSGZJuWhLuvGmYO6TDoV+Enc7ml5qM+qmnHmpD7fBmx092fcfQ/wbeCEVJkJ12fTk7y7X+zur3D3\nVwJ/Btzq7memit0InAlg49w8NV0aiTHZzmVmxxEuN32mVTEmln2ghS4lMLODgLcD/5Yq1tb6bDTO\ndtdprJMnzOxVcdRbgYdSxdpel43E2e66THkftZtA2l6fCTXjzEl9Pg68ycxeYmZG2O4Pp8pMuD4b\nveN1ysxsEfHmKXf/npmdZma/IN481ao4xpOMEfhTMzsHeAF4ntCW1w5HAN+x0CVEF7DG3b+fw/qs\nGyf5qNMLgDXxp/tG4Owc1mXdOMlHXWJmBxKOQD+UGJe7+qwXJzmoT3f/kZldD9wX47gXuHqq9amb\noURESkyP/xMRKTEleRGRElOSFxEpMSV5EZESU5IXESkxJXkRkRJTkhcRKTEleRGREvv/EU+cgf0d\nB5gAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1128814d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Let's plot the first two features together with the label.\n",
    "# Remember, while we are plotting the testing feature distribution\n",
    "# here too, you might not be supposed to do so in real research,\n",
    "# because one should not peek into the testing data.\n",
    "legend = ['rx', 'b+', 'go']\n",
    "pyplot.title(\"Training data distribution, feature 0 and 1\")\n",
    "for i in range(3):\n",
    "    pyplot.plot(train_features[train_labels==i, 0], train_features[train_labels==i, 1], legend[i])\n",
    "pyplot.figure()\n",
    "pyplot.title(\"Testing data distribution, feature 0 and 1\")\n",
    "for i in range(3):\n",
    "    pyplot.plot(test_features[test_labels==i, 0], test_features[test_labels==i, 1], legend[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, as promised, let's put things into a Caffe2 DB. In this DB, what would happen is that we will use \"train_xxx\" as the key, and use a TensorProtos object to store two tensors for each data point: one as the feature and one as the label. We will use Caffe2 python's DB interface to do so."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This is what the tensor proto looks like for a feature and its label:\n",
      "protos {\n",
      "  dims: 4\n",
      "  data_type: FLOAT\n",
      "  float_data: 5.40000009537\n",
      "  float_data: 3.0\n",
      "  float_data: 4.5\n",
      "  float_data: 1.5\n",
      "}\n",
      "protos {\n",
      "  data_type: INT32\n",
      "  int32_data: 1\n",
      "}\n",
      "\n",
      "This is the compact string that gets written into the db:\n",
      "\n",
      "\u0016\b\u0004\u0010\u0001\u001a\u0010�̬@\u0000\u0000@@\u0000\u0000�@\u0000\u0000�?\n",
      "\u0005\u0010\u0002\"\u0001\u0001\n"
     ]
    }
   ],
   "source": [
    "# First, let's see how one can construct a TensorProtos protocol buffer from numpy arrays.\n",
    "feature_and_label = caffe2_pb2.TensorProtos()\n",
    "feature_and_label.protos.extend([\n",
    "    utils.NumpyArrayToCaffe2Tensor(features[0]),\n",
    "    utils.NumpyArrayToCaffe2Tensor(labels[0])])\n",
    "print('This is what the tensor proto looks like for a feature and its label:')\n",
    "print(str(feature_and_label))\n",
    "print('This is the compact string that gets written into the db:')\n",
    "print(feature_and_label.SerializeToString())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Now, actually write the db.\n",
    "\n",
    "def write_db(db_type, db_name, features, labels):\n",
    "    db = core.C.create_db(db_type, db_name, core.C.Mode.write)\n",
    "    transaction = db.new_transaction()\n",
    "    for i in range(features.shape[0]):\n",
    "        feature_and_label = caffe2_pb2.TensorProtos()\n",
    "        feature_and_label.protos.extend([\n",
    "            utils.NumpyArrayToCaffe2Tensor(features[i]),\n",
    "            utils.NumpyArrayToCaffe2Tensor(labels[i])])\n",
    "        transaction.put(\n",
    "            'train_%03d'.format(i),\n",
    "            feature_and_label.SerializeToString())\n",
    "    # Close the transaction, and then close the db.\n",
    "    del transaction\n",
    "    del db\n",
    "\n",
    "write_db(\"minidb\", \"iris_train.minidb\", train_features, train_labels)\n",
    "write_db(\"minidb\", \"iris_test.minidb\", test_features, test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's create a very simple network that only consists of one single TensorProtosDBInput operator, to showcase how we load data from the DB that we created. For training, you might want to do something more complex: creating a network, train it, get the model, and run the prediction service. To this end you can look at the MNIST tutorial for details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The net looks like this:\n",
      "name: \"example_reader\"\n",
      "op {\n",
      "  output: \"dbreader\"\n",
      "  name: \"\"\n",
      "  type: \"CreateDB\"\n",
      "  arg {\n",
      "    name: \"db_type\"\n",
      "    s: \"minidb\"\n",
      "  }\n",
      "  arg {\n",
      "    name: \"db\"\n",
      "    s: \"iris_train.minidb\"\n",
      "  }\n",
      "}\n",
      "op {\n",
      "  input: \"dbreader\"\n",
      "  output: \"X\"\n",
      "  output: \"Y\"\n",
      "  name: \"\"\n",
      "  type: \"TensorProtosDBInput\"\n",
      "  arg {\n",
      "    name: \"batch_size\"\n",
      "    i: 16\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "net_proto = core.Net(\"example_reader\")\n",
    "dbreader = net_proto.CreateDB([], \"dbreader\", db=\"iris_train.minidb\", db_type=\"minidb\")\n",
    "net_proto.TensorProtosDBInput([dbreader], [\"X\", \"Y\"], batch_size=16)\n",
    "\n",
    "print(\"The net looks like this:\")\n",
    "print(str(net_proto.Proto()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "workspace.CreateNet(net_proto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The first batch of feature is:\n",
      "[[ 5.19999981  4.0999999   1.5         0.1       ]\n",
      " [ 5.0999999   3.79999995  1.5         0.30000001]\n",
      " [ 6.9000001   3.0999999   4.9000001   1.5       ]\n",
      " [ 7.69999981  2.79999995  6.69999981  2.        ]\n",
      " [ 6.5999999   2.9000001   4.5999999   1.29999995]\n",
      " [ 6.30000019  2.79999995  5.0999999   1.5       ]\n",
      " [ 7.30000019  2.9000001   6.30000019  1.79999995]\n",
      " [ 5.5999999   2.9000001   3.5999999   1.29999995]\n",
      " [ 6.5         3.          5.19999981  2.        ]\n",
      " [ 5.          3.4000001   1.5         0.2       ]\n",
      " [ 6.9000001   3.0999999   5.4000001   2.0999999 ]\n",
      " [ 6.          3.4000001   4.5         1.60000002]\n",
      " [ 5.4000001   3.4000001   1.70000005  0.2       ]\n",
      " [ 6.30000019  2.70000005  4.9000001   1.79999995]\n",
      " [ 5.19999981  2.70000005  3.9000001   1.39999998]\n",
      " [ 6.19999981  2.9000001   4.30000019  1.29999995]]\n",
      "The first batch of label is:\n",
      "[0 0 1 2 1 2 2 1 2 0 2 1 0 2 1 1]\n",
      "The second batch of feature is:\n",
      "[[ 5.69999981  2.79999995  4.0999999   1.29999995]\n",
      " [ 5.0999999   2.5         3.          1.10000002]\n",
      " [ 4.4000001   2.9000001   1.39999998  0.2       ]\n",
      " [ 7.          3.20000005  4.69999981  1.39999998]\n",
      " [ 5.69999981  2.9000001   4.19999981  1.29999995]\n",
      " [ 5.          3.5999999   1.39999998  0.2       ]\n",
      " [ 5.19999981  3.5         1.5         0.2       ]\n",
      " [ 6.69999981  3.          5.19999981  2.29999995]\n",
      " [ 6.19999981  3.4000001   5.4000001   2.29999995]\n",
      " [ 6.4000001   2.70000005  5.30000019  1.89999998]\n",
      " [ 6.5         3.20000005  5.0999999   2.        ]\n",
      " [ 6.0999999   3.          4.9000001   1.79999995]\n",
      " [ 5.4000001   3.4000001   1.5         0.40000001]\n",
      " [ 4.9000001   3.0999999   1.5         0.1       ]\n",
      " [ 5.5         3.5         1.29999995  0.2       ]\n",
      " [ 6.69999981  3.          5.          1.70000005]]\n",
      "The second batch of label is:\n",
      "[1 1 0 1 1 0 0 2 2 2 2 2 0 0 0 1]\n"
     ]
    }
   ],
   "source": [
    "# Let's run it to get batches of features.\n",
    "workspace.RunNet(net_proto.Proto().name)\n",
    "print(\"The first batch of feature is:\")\n",
    "print(workspace.FetchBlob(\"X\"))\n",
    "print(\"The first batch of label is:\")\n",
    "print(workspace.FetchBlob(\"Y\"))\n",
    "\n",
    "# Let's run again.\n",
    "workspace.RunNet(net_proto.Proto().name)\n",
    "print(\"The second batch of feature is:\")\n",
    "print(workspace.FetchBlob(\"X\"))\n",
    "print(\"The second batch of label is:\")\n",
    "print(workspace.FetchBlob(\"Y\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
